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" Man built most nobly when limitations were greatest 


- Frank Lloyd Wright 





Population being or online 




Everyone running its state! 
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The MSM 


Controllers 


o Game Pad 
o Behavior Tree 
o Path Follower 
o Vision System 
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Controllers 


Sensors 


o Per System 
o Global I/Os 
o Partially Replicated 
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class CPlayerCoveredMotionStateDef : public COnFootMotionStateDef 

{ 

DECLARE_STATE_DEF_PARAMS(CPlayerCoveredMotionStateParams) ; 
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DECLCRE_ST/- TE_DE ~_BEGIE. (CPlayerCoveredMotionStateDef, COnFootMotionStateDef ^ 

DECLARE_STATE_DEF_5UBSTATE(Approach) 

DECLARE__STATE_DEF_SUBSTATE(Enter) 

DEC LAR E_STATE_DE F_SUBSTATE (Idle) 

DEClARE_STATE_DEF_SUBSTATE(Start) 

DECLARE_STATE_DEF_SUBSTATE ( M&ue) 

DE CLAR£_ST AT E_DE F_SUBSTAT E ( S t op ) 

DECLARE_STATE_DEF_SUBSTATE(TurnAnticipation) 

DECLARE_STATE_DEF_SUBSTATE(Turn) 



/ 


class CPlayerCoveredMotionStateParams : public CBaseStateParams 

{ 

DECLA c E__STA i E_P^RAf ;5(CPlayerCoveredMotionStateParams, CBaseStateParams) 


eStatePriorityJiigh) 



DECLr RE_NETDAT c BEGIN(CPlayerCoveredMotionStateParams, CBaseStateParams : :NetData) 

L EC E_.\ 5 t e :• T -i__f E (m_coverEnt ryAttachment , CCoverAttachment, CNetDataCoverAttachment ) 

(m_randomValue J ndFloat, storm: : DataFloatcl, storm: : £ t E r . r a F 1 c a t E p i 1 c n_C_ . 

DEC.-- E_. ETD J _ (m_skipEntry i ndBool, storm: :DataBool) 

DECLARE_NETDATA_END( ) 



CStateRequest<CPlayerCoveredMotionStateDef> coverRequest ; 

coverRequest .GetParams( ) .SetCove^EntryAttachment (coverAttachment ) ; 
coverRequest .GetParams( ) . SetRandoirValue(ndRandFloat ( ) ) ; 

coverRequest . Send( ); 
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The MSM is running tracks 
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The MSM is running tracks 


Motion moujan/ 

ex : Standing , Cover, Driving, Swimming 


Action /'aek.Jan/ 

ex: Climbing, Reacting, Fighting, Interacting 
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ex : Looking, Protecting, Aiming 
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ex: Firing, Throwing, Flacking, Masking 
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The MSM is running tracks 





















The MSM is running tracks 
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The MSM is running tracks 
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The MSM is running tracks 

















The MSM is running tracks 
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The MSM is running tracks 


















Motion Fundamentals 
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Root motion approach for fluidity 


Animation driving displacement 

■ mostly translations 

■ sometimes rotations 




Root motion approach for fluidity 




Animation driving displacement 

■ mostly translations 
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■ sometimes rotations 





pencil shape 


Heading is the intention 

■ expressed by head direction 

■ read from angular input 

■ high frequency filtering 


Facing is body lagging 


input 


Direction segmentation for fluidity 











Displacement sliding for responsive precision 





Facing is body lagging 

■ expressed by shoulders direction 

■ read from lateral input 

■ high frequency filtering 


Displacement is sliding 





Data markup for responsive precision 


Custom entry in animations 
Adaptive pose matching 
Latency reduction 


Movement : 
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Data markup for responsive precision 



■ Custom entry in animations 

■ Adaptive pose matching 

■ Latency reduction 
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Data markup for responsive precision 


■ Custom entry in animations 

■ Adaptive pose matching 


Latency reduction 
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Data markup for responsive precision 



■ Custom entry in animations 

■ Adaptive pose matching 

■ Latency reduction 




Data markup for responsive precision 


■ Custom entry in animations 

■ Adaptive pose matching 

■ Latency reduction 
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Jog Stop Right Foot 
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Data markup for responsive precision 


■ Custom entry in animations 

■ Adaptive pose matching 

■ Latency reduction 
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Data markup for responsive precision 


Custom entry in animations 
Adaptive pose matching 
Latency reduction 
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Pose matching transitions navigation 



VIDEO EXAMPLE 


Data segmentation for start precision 


■ Optical illusion : Head motion reactivity 

■ Read mind for direction & speed : Fixed delay 

■ Anticipation of ~6@8 frames (playtest) 
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Data segmentation for start precision 






■ Optical illusion : Head motion reactivity 

■ Read mind for direction & speed : Fixed delay 

■ Anticipation of ~6@8 frames (playtest) 

■ Same philosophy applied to plants & turns 





ANTICIPATION 

(common) 


TRANSITION 

to idle , to walk, to jog, to sprint 



Action Fundamentals 
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Dedicated sensors for contextual climbing 


Physical shape/ray sensor 

■ Real-time & Functionnal, but... 

■ Expensive on many characters 

Edge annotation sensor 

■ Offline generated & "Precise" 

■ Cheap with some filtering 














Object contour [Annotations generation] 
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BODY 


Original animation 



BODY 
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Original animation 
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Data markup for seamless sequence 
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Original animation 
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Dedicated sensors for contextual reactions 



Stimulus sensor 

■ Catching impacts, explosions, etc... 

■ Managing reactions & impulses 

Body sensor 

■ Acquiring bone poses 

■ Computing body directions 






Data presets for contextual reactions 


Presets for required animations 
Custom data for each ragdoll bone 


Rigidbody Settings 


Bone Settings 


Max Linear velocity 
Max Angular velocity 
Velocity Damping 


Stiffness 

Collision 
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Simulation for seamless realization 




Impact anticipation for seamless ragdoll blend 
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Bike ejection trajectory simulation 
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Seamless sequence with interruptions 


Shotgun Ammunition: 8 
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Stance Fundamentals 
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Pose modification layering for diversity 


Emphasize emotion diversity 
Break idle stillness 


Characters changing stance 

object in hands 

context 

mood 


Pose modification layering for diversity 




Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 
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Pose modification layering for diversity 


Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 



Pose modification layering for diversity 


Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 


Pose modification layering for diversity 


Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 



Pose modification layering for diversity 


Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 
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Eyes Compensation 


Data driven look-at layering for consciousness 


(3D) look at point 


Feet Blend Space Computing 


Inject Torso Pitch/Yaw RTCP 


UpperBack Blend Space Computing 


Inject Neck Pitch/Yaw RTCP 


Inject Head Pitch/Yaw RTCP 
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Feet Blend Space Computing 

// Get current referential 

CQuaternion CRightFoot( sourceResult->m_displacementTransform.GetRotation( ) 

* (modelCommonRaw->Get‘'odelTransfcrT( rightFootldx, skeletonRaw, jointTransformArray ) .GetRotation( ) InvTSRightFoot ) ); 

CQuaternion CLeftFoot ( sourceResult- >m_displacementTransform.GetRotation( ) 

* (modelCommonRaw->Get M odelTransfor r r( leftFootldx, skeletonRaw, jointTransformArray ) .GetRotation( ) * InvTSLeftFoot ) ); 
CQuaternion feet Rot; 

feetRot .QuaternionSlerp( CRightFoot, CLeftFoot, 0.5f ); 
feetRot . RemoveAllExceptA:<is( CVector3( 0.f, 0.f, l.f ) ); 


UpperBack Blend Space Computing 

// Get current referential 

ndQuat upperBack ( sourceResult- >m_displacementTransform.GetRotation( ) 

* (modelCommonRaw->Get^cdelT ansfo^m( upperBackldx, skeletonRaw, jointTransformArray ) .GetRotation( ) * InvTSUpperBack) ); 


m_lookatWorldTransform = ndPosQuatTransform( upperBack, neckPos ); 




Data driven look-at layering for consciousness 


Contextual data scalability 

■ distinction between glance & stare 

■ in car head look-at recycling 

■ anti-look-at for fire protection 

■ sky is the limit... 

(within animator & memory constraints) 
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Kinesic Fundamentals 
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Body Awareness 


Game Feel 


Contacts with objects for connectivity 


Enhance the power of touching 
pushing doors 
grabbing grenades 
putting on the mask 
getting hands in pockets 
...or even say hello to strangers! 
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flexing knees 















Recoil layered system 
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" All fine architectural values are human values , else not valuable ” 

- Frank Lloyd Wright 





Questions ? 


More ? Meet me @ the Ubisoft lounge! 


Thursday 


12.30PM @ 1.30PM 
West Hall, 3 rd Floor 


You can find the presentation's slides with some notes below... Cheers! © 
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We are not here only for going to GDC parties... 

We are here for sharing experience... 

But also, most of all to enhance players' experience in the end! 




The player's experience can be built by multiple things... 

But one of the most important is the immersion quality factor! 
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The player's experience can be built by multiple things... 

But one of the most important is the immersion quality factor! 




And immersion can be defined by two high level topics... 

The narrative aspects of the game, which are not the focus of the presentation today 
And the game feel of the game, which is going to be the hot topic of this talk! 







mmersion 


Usability 


Grounding 


Spatial Awareness 


Body Awareness 


And the game feel can be divided into four big categories : 

• Usability : for the fluidity and the precision of things 

• Grounding : for the contextual and seamless activities 

Spatial Awareness : for the consciousness and the diversity of the environment 

• Body Awareness : for the realism and the connectivity of the character itself 

Remember these four categories because everything concerning the Watch_Dogs 
state machine we will talk about today will be linked to these! 
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Before diving into the architectural details of Watch_Dogs state machine let's keep 
mind this quote from a famous architect... 




Our first limitation is that open-world games these days need to handle an orgy of 
data of all sorts! 
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Our second limitation is that open-world games these days need a good density of 
population to boost immersion quality! 
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Our third limitation is that open-world games these days need to be online to live the 
game with friends live! 

So, we need a lot of characters doing diverse credible things in the simulated world 
coming from a local instance or from a remote instance in our peer to peer 
architecture... 
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In the end. ..everyone is running its own state machine! 
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The Multitrack tate achine 


MSM 
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During Watch_Dogs, we decided to build a custom Multitrack State Machine, which 
is going to be called the MSM. 
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Controllers (feeding inputs to the state machine at the beginning of the frame) 

• Game Pad inputs 

• Behavior Tree tasks 

• Path Follower instructions 

• Vision System computations 
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MSM 


The MSM 


Controllers 

o Game Pad 
o Behavior Tree 
o Path Follower 
o Vision System 


Controllers (feeding inputs to the state machine at the beginning of the frame) 

• Game Pad inputs 

• Behavior Tree tasks 

• Path Follower instructions 

• Vision System computations 




Sensors (feeding inputs to the state machine asynchronously during the frame) 

• Frontal Collisions reports 

• Cover Planes targets 

• Jump Annotations possibilities 

• Stimulus Emissions events 




MSM 


The MSM 


Controllers, 


Sensors 


o Frontal Collisions 
o Cover Planes 
o Jump Annotations 
o Stimulus Emissions 


Sensors (feeding inputs to the state machine asynchronously during the frame) 

• Frontal Collisions reports 

• Cover Planes targets 

• Jump Annotations possibilities 

• Stimulus Emissions events 




State Pool (being the definition of the state machine) 

• One pool per character which is defining its state domain 

• One and only one single instance of each state in the pool for hypothesis & 
executions 

• Causing less memory fragmentation by keeping the footprint pretty stable once 
the character is initialized 


17 



The MSM 



State Pool (being the definition of the state machine) 

• One pool per character which is defining its state domain 

• One and only one single instance of each state in the pool for hypothesis & 
executions 

• Causing less memory fragmentation by keeping the footprint pretty stable once 
the character is initialized 




Data Containers (being the input/output storage of the character status) 

• One container per system sharing various information like speeds, positions, 
object handles, etc... 

• Allowing systems like the state machine to read or write this status information to 
run 

• Replicated partially because we want to minimize the bandwidth usage in 
between hosts & replicas 
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The MSM 



Data Containers (being the input/output storage of the character status) 

• One container per system sharing various information like speeds, positions, 
object handles, etc... 

• Allowing systems like the state machine to read or write this status information to 
run 

• Replicated partially because we want to minimize the bandwidth usage in 
between hosts & replicas 




Now let's look at a simple example about how these elements collaborate with each 
other to make the MSM run 

Controllers & Sensors are able to send asynchronous state requests 

- State request object built locally and then sent by event to the MSM 

- State request object made of the state definition which is a lightweight structure 
of the state 

- State request object also made of the start up parameters filled by the requesting 
code 
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Once these state requests are stacked inside the MSM, they will be consumed at the 
beginning of the frame 

- First pass of consumption is priority sorting using the state definition object within 
the request object 

- Second pass of consumption is priority comparison with the current state running 
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Once they are well sorted in between themselves, we proceed with their validation 

- First step is extracting the start up parameters stored in the state request object 

- Second step is pushing these start up parameters to the corresponding state in the 
pool 

Every single instance of these states have a const validating function to evaluate the 
parameters even if the state is currently running 

Because we keep the same start up parameters object from the beginning of the 
request, the state can cache expensive computing in it also 
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State Execution 





Once the highest priority state passed it is elected as the running state of the MSM 

During its execution it will use the necessary data containers to read/write useful 
information to be able to run properly. And while it is being executed, the ecosystem 
will continue to request other states 
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Let's dig deeper into the internal state mechanics... 
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Phone? Fence Edge 


One particularity of this multi track state machine is that every state doesn't have 
infinite depth like a hierarchical state machine 

Deliberately forced only 1 layer of sub states inside each state and common services 
in between states are contained within the class hierarchy 

Every sub state can be or not referencing a dynamic animation tree from which we 
build the resource dependencies for each state pool 

Decision trees are logical trees and Blend trees are leafs of them 
Substates are responsible to push proper real-time control parameter structure 
Decision trees and blend trees evaluate the proper animations to play based on this 
RTCP data structure 

Substates are also responsible to push environmental information called anchors 
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So when the state starts, it receives the start up parameters to decide what to do... 

Then the good sub state will run with precise information from them 

At first we had a graphical tool to express states & substates possible transitions 
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But in the end, it turned out to be a total mess to understand because everything 
was linked to everything... 

Mainly because we want to be easily interruptible but also because when you 
replicate a state you might miss network packets! 


28 



.. - 


State Coi 

: ub- tates 

Xs* ' 




• 

Wm 

fCZ p 


— 


ss ) n __ _ 


Ww 

B 

ss ss 



MS 

ss 

IB 

, # 4 


ss 


/ 

m 

SS 55 

I mmt\ 



* 


1 * 






r*TC ‘;:r *TT 


’IfcvV 

PF 





Finally free for all approach 


So we decided to take a free for all approach by removing this useless structure to 
virtually link them all together 
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State Co o : ub- tates 



2014 


As a concrete example, here is the cover state graph with all its substates in 2012... 
And then we started removing useless internal links during 2013... 

To finally completely remove this graphical tool from our state machine pipeline in 
2014... 

In the end, good programmers are lazy by definition. ..They don't want to manage a 
graphical state machine if they have to manage the code state machine on the other 
side! 
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So we removed all these links because we needed to network efficiently the MSM. 

There are 3 main things we need to replicate to achieve a "kinda" deterministic state 
machine : 

- Start up parameters to synchronize the state activation 

- State & Substate unique IDs to drive the internal flow 

- Data containers partial data replication system 

Good programmers are lazy! So we created a way to abstract the replication process 
the most possible... 

By default, the whole start up parameters will be automatically packed and sent to 
replica's machines 

And in parallel, the data containers will compress and send tagged members only to 
the replica's machines 

This is done by a special macro system within the C++ to auto generate the accurate 
members for the network object model. A float is a float for the gameplay engine 
layer but it is a compressed resolution float for the network layer 
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CPlayerCoveredMotionStateDef : public COnFootMotionStateDef 
ECLARE_STA 7 E_C 5 F_PA ' 5(CPlayerCoveredMotionSt ateParams ) ; 

EC. a 3E_ST^ I E_DEF_BEGI.’>( CPlayerCoveredMotionStateDef, COnFootMotionStateDef, 

D£CLARE_STATE_DEF_SUBSTATE (Approach) 

DECLARE_STATE_DEF_SUBSTATE( Enter ) 

DECLARE_STATE_DEF_SUBSTATE(Idle) 

DECLARE_STAT E_DEF_SUBSTATE (St art ) 

DECLARE_STATE_DEF_SUBSTATE(Move) 

DECLARE_STATE_DEF_SUBSTATE(Stop) 

DECLARE_STATE_DEF_SUBSTATE(TurnAnticipation) 

DECLARE_STATE_DEF_SUBSTATE(Turn) 


m 


| class CPlayerCoveredMotionStateParams : public CBaseStateParams 

{ 

DECL^ : <5_STATE_A-« - S(CPlayerCoveredMotionStateParams, CBaseStateParams) 

DECLARE^ NETDATA_E EC- 1 ’.(CPlayerCoveredMotionSt ateParams, CBaseStateParams : : Net Data) 

(m_coverEntryAttachment , CCoverAttachment, CNetDataCoverAttachment ) 

(m_randomValue, ndFloat, storm: :DataFloat<l, storm: 

(«i_skipEntry, ndBool, storm: :DataBool) 

>ECLARE_NETDATA_END( ) 


CStateRequest<CPlayerCoveredKotionStateDef> coverRequest ; 

coverRequest .6etParams( ) .SetCoverEntryAttachment (coverAttachment ); 
coverRequest . Get Par ams ( ) . Set RandomValue ( ndRandF loat ( ) ) ; 


coverRequest . Send ( ) ; 


Here is a small glimpse at how things work automatically in the code for the motion 
cover state for example : 

- At the top, straight forward sub state definition with one liners 

- In the middle, straight forward start up parameters networking with one liners 

- At the bottom, straight forward state request generation & emission with one 
liners 
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And then. ..you'll need good tools to debug your animations playing... 

The best one we used on Watch_Dogs was called the Ghost feature 
Creating a local replica of the player using the local machine network loopback 

That way you can easily simulate random lags, packet losses and see both entities in 
the same screen! 
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Since the beginning we are talking about a multi track state machine... But where and 
what are these tracks I? 
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The MSM is running tracks 
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Another limitation we forced was to cap the number of tracks to 4 and to dedicate 
each one of them to a responsibility 
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The MSM is running tracks 



Another limitation we forced was to cap the number of tracks to 4 and to dedicate 
each one of them to a responsibility 
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ex : Climbing, Reacting, Fighting, Interacting 
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ex : Firing, Throwing, Hacking, Masking 

.. js. 


The first one is called Motion, move the character from A to B 

The second one is called Action, do things requiring the whole body 

Motion & Actions are sharing the full body layer within the animation engine, which 

means they play one dynamic tree at the time together 

The third one is called Stance, convey the context of the character 

The last one is called Kinesic, represent hand manipulations 

Stance & Kinesic are sharing another layer on top of the full body one within the 

animation engine 

This structure is highly inspired on how humans actually think. ..moving efficiently and 
doing an action efficiently is not something that fits well simultaneously 
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The MSM is running tracks 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 


38 



• : «rt The MSM is running tracks 

A 


• 

S3 

Motion Standing 

fr? 



(pP 



, / A 

© 

Action Q 

/ 

©) 

Stance £ 

N.’ \ 

© 


ftp 

Kinesic Reload 




Let's have a look at a simple sequence example illustrating the inter track roles & 
services 
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The MSM is running tracks 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 

Climbing decided at a certain point during it's climbing over animation to disable 
anything related to Kinesic since hands are being used. 

This mechanism is a simple bit mask property that motions & actions can modify 
dynamically to allow or stop incoherent layered animations with them. 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 

ATM usage asked the Motion track to go to a specific location to start the interaction 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 
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Let's have a look at a simple sequence example illustrating the inter track roles & 
services 

Again, hands are be used with the ATM so the bit mask removes any possible Kinesic 
states but at the same time, a sexy lady will pass by and trigger the look at state over 
the end of the ATM animation. 
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In between themselves, the top track of the pair can overlay, pause or halt the 
underlying one 


46 




The rest of the talk will be split into 4 sections showing some concrete examples of a 
couple of states of each of these tracks. 

Let's start with the Motion track which is supporting the usability aspect of the game 
feel. 

Keywords to keep in mind while building these states are fluidity and precision! 
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To enforce the maximum fluidity of movement possible we will use a root motion 
approached based on a heavy load of motion capture data. 

Animations are going to drive the collision displacement mostly in translation 
because we will want to keep the rotation for the code side since the shape is pretty 
much cylindrical 
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Speaking of the displacement collision, we tried various approaches like a : 

- Cylinder shape, which was not good for auto-stepping sidewalks 

- Capsule shape, which was not good for steep steps to climb 

- Pencil like shape, which was good for most cases but slightly glitchy sometimes 
with lateral noise on sidewalks but it was compensated by putting a spring 
damping between the root of the animation skeleton and the displacement 
location 
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Direction segmentation for fluidity 


Heading is the intention 

■ expressed by head direction 

■ read from angular input 

■ high frequency filtering 


Facing is body lagging head|ng 


facing 


We said earlier that root motion rotations were handled by the code, mostly because 
we wanted to decouple them into a chain : 

- Heading : optical illusion given by the head fast rotation towards the direct input 
of the joystick 

- Facing : the rest of the body lagging behind with a sequential chain of dampers 
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Here you can see the head leading the movement being almost 1 to 1 with the 
joystick direction and then the body will catch up afterwards to keep everything fluid 
with some natural inertia 
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Displacement sliding for responsive precision 




Facing is body lagging 

■ expressed by shoulders direction 

■ read from lateral input 

■ high frequency filtering 


Displacement is sliding 


We said earlier that translations were going to be handled totally by animations but 
we lied. ..There a case here it is useful to make the displacement slide artificially : 
With facing lagging means we can read the angular acceleration of the joystick to 
inject some lateral movement to the displacement to dodge something and give the 
optical illusion of the shoulders leading the movement hiding completely the feet 
sliding on the ground 
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With abrupt joystick inputs given we can move the displacement towards the desired 
direction and on top of that we can use a multi-dimensional parametric blend to fit 
the movement properly 
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To continue to talk about precision, let's talk about animation mark up. Let's take feet 
mark ups category for example which are generated by an automatic detection 
algorithm for LF, RP, RF & LF. The key here is to never wait for the proper mark up to 
happen, you have to blend as soon as it is needed for the player. 
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Let's say we are playing a jog loop parametric blended animation and suddenly the 
player wants to sprint. 
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Let's use a pose matching to analyse where we are between the main feet poses to 
match the same pose pair in the following animation. That way we can score every 
potential pairs about their proportion and find a winner spot to blend in. 
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Data markup for responsive precision 



■ Custom entry in animations 

■ Adaptive pose matching 

■ Latency reduction 




This can also be applied to transitions animators are going to plug in between 
specific loops to enhance the feeling of inertia and speed build up. We are going to 
pose match the transition first to then pose match the future animation 
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Pose matching is also used between loops and stop animations for example. In this 
case, we will have multiple stop animations to choose from. One starting with the left 
foot and the other starting with the right foot. The scoring function will consider the 
good pair, the proportions of this pair and the frame location of this pair also to 
choose a winner. 
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Pose matching is also used between loops and stop animations for example. In this 
case, we will have multiple stop animations to choose from. One starting with the left 
foot and the other starting with the right foot. The scoring function will consider the 
good pair, the proportions of this pair and the frame location of this pair also to 
choose a winner. 

On top of that, you can extend the pose matching to multiple pose categories like 
arms mark up or other special mark up you want. In the end, you will have to tweak 
the accumulation of these categories' scores weight to select the proper animation 
to play. 
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We are often breaking animations to play another but there are some sweet spots to 
add some acting on the character sometimes. That's why most non-looping 
animations will have a settling portion after the crucial portion and this seperation 
will be defined by the safe cut mark up. That way, we can give some contextualism to 
the character fitting with what previously happened and also say to the code that 
from now on, this animation is 100% interruptible by anything. 
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Let's have a look at various speed changes using pose matching through data driven 
transitions 
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Data segmentation for start precision 
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Optical illusion : Head motion reactivity 
Read mind for direction & speed : Fixed delay 
Anticipation of ~6@8 frames (playtest) 



Finally, to boost the precision and the responsiveness of the start animations we will 
use the trick of the head motion reactivity because we don't know when the player 
starts to push the joystick at which force and in which direction it will end. We can't 
read the mind of the player. ..so playtests showed we needed between 6 to 8 frames 
at 30 fps to know if the start animation will finish to idle for a turn on spot, to walk, 
to jog or to sprint. 
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Data segmentation for start precision 


Optical illusion : Head motion reactivity 
Read mind for direction & speed : Fixed delay 
Anticipation of ~6@8 frames (playtest) 

Same philosophy applied to plants & turns 



So we separated the start into 2 sub states : 

- One for the common anticipation of all the endings possible : moving mostly the 
head 

- One for the desired ending at the correct speed and the correct angle 

The same concept has been applied to plant & turns to be able to anticipate and to 
be reactive 

The same approach was used when we built the cover system 
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Here we can see various examples of starts & plants triggered and interrupted with 
various endings 
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Let's follow with the Action track which is supporting the grounding aspect of the 

game feel. 

Keywords to keep in mind while building these states are seamless and contextual! 
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The first sensors developped were shape and ray casts 

Functional, but can be hard to entirety obstacles properly 
Expensive if used by many characters 

The second sensors developped were offline data generated ones 
Scan game world to detect ledges of every static object 
Same information used on static and dynamic objects 
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These offline generated annotations are built from the collision meshes in the static 
world 
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To afterwards do the voxelization of the world to build a voxel height field 
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From which we will extract surfaces with proper filters and proper thresholds 
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To obtain in the end all useful object contours from which we will create the 
annotations 
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All these annotations are going to be stored into a spatial hash map for optimal 
search time since we have thousands of annotations per world sector. On top of that, 
we will have dynamic objects adding their own annotations in local space to the 
world 


The first advantage == Performance 


to avoid having too much 


data overlapping and check clearance for the character 
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In the end the scanning was so cheap that added lateral scanning to boost the 
contextualization of jumping animations. That way we were able to find walls or 
objects on the side the support the climb over animation in that case. 


72 




Here is a quick look at how it behaved in-game for a fence surrounded by static walls 
or dynamic objects 




Data segmentation for seamless sequence 
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One single mocap clip : Seamless action 
One single animation scene : Easier editing 
Multiple sub-states logic for scalability 



We previously talked about data segmentation in the motion track section and here 
again we will try to split the data to fit our needs in terms of seamless playback into 
the world. 

For the climbing on animation for example, we will use one raw motion captured 
animation to ease the editing for the animator but we will split it into 3 different sub 
states for logic purposes. 
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For that example, it's split into 3 parts, the transition in, the body and the transition 
out 

Some other jumping actions might add an approach sub state before the transition in 
or a landing sub state before the transition out 
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Each of these specific sub state will have a responsibility to make this single 
animation fit the imperfections of the world. 

Tranln for example compensate the fact that the original animation was meant to 
start 1 meter away from the wall on the XY plane and it was also supposed to start 
totally forward without any angles. But the mark up in this animation will allow us to 
blend to this synchronized anchor which will compute the delta between the reality 
and the original location. This error, in translation and in rotation will be distributed 
on all the frames of the Tranln portion. But not in a linear way, it will inject this error 
compensation on each frame based on its relative proportion to the total 
displacement done during this segment. That way the animation will be fully sync to 
start the body segment. 
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Body will then do the same but with the Z axis since the animation was meant for a 
lm high wall instead of a 1.3m. During the whole segment, the error will be 
distributed to arrive flush at the top with the proper hand placement. 
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TranOut fades out all these synchronized anchors pretty rapidly to give back 
progressively as soon as possible the control to the player for its direction and speed 
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Data markup for seamless sequence 
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We briefly talked about the settling portion of the animations during the motion 
track part and here again, the transition out segments will be a good place to play 
some nice acting if the player is not touching the joystick. 

But this safe cut mark up is also going to be useful to tell the code that it can be 
interrupted to trigger another jumping action since annotations could be pretty close 
to each other. So instead of settling, we will interrupt and trigger back to back the 
next one. We will reuse the pose matching algorithm to enter at the proper frame to 
be as seamless as possible. 
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We briefly talked about the settling portion of the animations during the motion 
track part and here again, the transition out segments will be a good place to play 
some nice acting if the player is not touching the joystick. 

But this safe cut mark up is also going to be useful to tell the code that it can be 
interrupted to trigger another jumping action since annotations could be pretty close 
to each other. So instead of settling, we will interrupt and trigger back to back the 
next one. We will reuse the pose matching algorithm to enter at the proper frame to 
be as seamless as possible. 
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Here is an example of how it looked in game when chaining multiple jumping actions 
back to back... 
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Dedicated sensors for contextual reactions 
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Stimulus sensor 

■ Catching impacts, explosions, etc. 

■ Managing reactions & impulses 

Body sensor 

■ Acquiring bone poses 

■ Computing body directions 



A second example for the action track will be the ragdoll reaction state 

Which is using some impacts sensors to listen to bullet hits, impacts, explosions and 
stuff like that. 

And with this, we are going to have some body sensors to know how the body is 
placed, oriented and moving. 
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Data presets for contextual reactions 
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Using the Havok ragdoll, we built a preset system for animators to let them tweak 
properly the ragdoll for any of their animations with custom configurations : 

- Ragdoll type for rigid body controller or powered constraint for example 

- Stiffness or collision enabled of the bones 

- Velocities or damping for the bodies 

That way they control better what will be triggered when blending from animation 
to physics 
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Ragdolls are mostly used in the industry for projection in the air and it can lead to 
special organic situations sometimes that are difficult to tweak to get an appealing 
result 
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imulation for seamless realization 



Instead of going fully procedural, we decided to go heavily data driven with a 
trajectory system instead 
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When receiving the impact stimulus, we compute right away the forces, the 
directions to start a ballistic simulation of the character propulsion in the air. We will 
split this ballistic curve anticipation with a certain resolution to shape cast along it 
to see where it is going to hit something. That way, we can predict in advance where 
the body will end up. 
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Simulation for seamless realization 
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Once we have that, we can launch the crafted parametric blended animation and 
move it along the computed ballistic curve. The flying sub state will be responsible of 
pushing the RTCP describing the flight like the duration, the length, the apex, the 
angles, etc... 

At the same time we launch the animation, we activate the rigid body controller of 
the ragdoll using the presets tweaked by the animator which are ultra stiff for the 
flight pose. We also continue to cast a bit forward along the line to detect dynamic 
objects because we need a bit of anticipation to blend properly between the 
animation and the ragdoll. 
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Being in the proper pose, we will be able to blend from the rigid body controllers to 
the get up animation fitting the landing pose and we will progressively give back the 
control to the player to move again 
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Impact anticipation for seamless ragdoll blend 



When anticipating an impact we will blend the powered constraint ragdoll based on 
the animation ragdoll preset and let it go to collide with the environment while 
trying to do a protecting pose. 

Once stabilized, we will have to switch to ground pain to blend from the powered 
constraint to the rigid body controllers matching a known get up pose. This is done 
by checking the orientation of some ragdoll bodies like the torso ones since limbs 
will be replaced more easily by the Havok controllers. 

Once in the proper pose, we will be able to blend from the rigid body controllers to 
the get up animation fitting the analyzed pose and we will progressively give back 
the control to the player to move again 
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Could be used for motorcycle crash ejection! 
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Or car bail out ejection! 
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Or, of course, grenade explosion propulsion! 


92 




Which can be interrupted by dynamic objects! But in most cases, we will be able to 
fully control the look of it from the beginning to the landing without even activating 
the powered constraint ragdoll. 
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Here is a sequence of the game played with grenades while being invincible to 
showcase multiple angles and ballistic predictions and multi-dimensional parametric 
blended animations. 
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Let's follow with the Stance track which is supporting the spatial awareness aspect of 
the game feel. 

Keywords to keep in mind while building these states are diversity and 
consciousness! 
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We are talking about diversity because we want to feel the various context of 
characters. 

This can be influenced by objects in hand, the mood or the gameplay context... 

To modify the pose is really useful to emphasize the emotion diversity and to break 
the idle stillness of characters 
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Pose modification layering for diversity 



Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 



Most of these pose modifications are done with layered additive animations playing 
on top of the full body 

We expose different weights for each bone in the skeleton and we also expose the 
possibility to change the bone weights at different speed to change the way we are 
going to alter the full body animation. If you have already tried playing with additive 
animation below the root, you know that feet might go through the ground and this 
is why Feet IK is not only useful for slopes or to prevent feet from sliding... It is really 
useful to apply additive data on top of the legs and then resolve the IK for the 
original locations of the feet. 
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Pose modification layering for diversity 



Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 



Most of these pose modifications are done with layered additive animations playing 
on top of the full body 

We expose different weights for each bone in the skeleton and we also expose the 
possibility to change the bone weights at different speed to change the way we are 
going to alter the full body animation. If you have already tried playing with additive 
animation below the root, you know that feet might go through the ground and this 
is why Feet IK is not only useful for slopes or to prevent feet from sliding... It is really 
useful to apply additive data on top of the legs and then resolve the IK for the 
original locations of the feet. 
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Most of these pose modifications are done with layered additive animations playing 
on top of the full body 

We expose different weights for each bone in the skeleton and we also expose the 
possibility to change the bone weights at different speed to change the way we are 
going to alter the full body animation. If you have already tried playing with additive 
animation below the root, you know that feet might go through the ground and this 
is why Feet IK is not only useful for slopes or to prevent feet from sliding... It is really 
useful to apply additive data on top of the legs and then resolve the IK for the 
original locations of the feet. 
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Pose modification layering for diversity 



Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 



Most of these pose modifications are done with layered additive animations playing 
on top of the full body 

We expose different weights for each bone in the skeleton and we also expose the 
possibility to change the bone weights at different speed to change the way we are 
going to alter the full body animation. If you have already tried playing with additive 
animation below the root, you know that feet might go through the ground and this 
is why Feet IK is not only useful for slopes or to prevent feet from sliding... It is really 
useful to apply additive data on top of the legs and then resolve the IK for the 
original locations of the feet. 
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Pose modification layering for diversity 


Additive layering on full body 

■ different weights for each bone 

■ bone weights different for each speed 

■ feet IK resolution post layering 

■ ...not only useful for slopes! 


Most of these pose modifications are done with layered additive animations playing 
on top of the full body 

We expose different weights for each bone in the skeleton and we also expose the 
possibility to change the bone weights at different speed to change the way we are 
going to alter the full body animation. If you have already tried playing with additive 
animation below the root, you know that feet might go through the ground and this 
is why Feet IK is not only useful for slopes or to prevent feet from sliding... It is really 
useful to apply additive data on top of the legs and then resolve the IK for the 
original locations of the feet. 
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Enough for the pose modifications diversity, let's look at the technique we developed 
to enhance the consciousness of the character. 

Initially we started by coding a procedural approach to do a look-at : 

- Tried model space since it works well with upright spine characters but it was not 
looking good in crouch stance 

- Tried local space to fix the crouch stance problems but it was still looking a bit 

uncanny 

- Tried to blend both ideas but ended up with a lot of use cases to tweak in the end 
without any WOW visuals 

Finally cried. ..decided to go fully data driven while using custom made spaces for the 
blending 
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The way we built the data for the look at is that we split the body into multiple 
sections and each of these sections are going to be dedicated to a portion of the look 
at. In our case, it is split into 3 parts, the torso, the shoulders/neck and the head. The 
evaluation of the 3D world anchor for the point of interest will be done from the 
bottom to the top. Starting with the torso animation node to compute the 
proportion of its rotation in pitch & yaw that are going to be RTCP internally 
injected. And then the shoulders are going to be computed right after, continuing 
from where the torso stopped and converge a bit more based on the defined 
proportions and finally, the head, with the eyes, will complete the job at each frame. 
For each of these sections, the animators created a multi-dimensional parametric 
blended animations fitting within human constraints to move the proper bones for 
this section all around the required pitch & yaw. 
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The way we built the data for the look at is that we split the body into multiple 
sections and each of these sections are going to be dedicated to a portion of the look 
at. In our case, it is split into 3 parts, the torso, the shoulders/neck and the head. The 
evaluation of the 3D world anchor for the point of interest will be done from the 
bottom to the top. Starting with the torso animation node to compute the 
proportion of its rotation in pitch & yaw that are going to be RTCP internally 
injected. And then the shoulders are going to be computed right after, continuing 
from where the torso stopped and converge a bit more based on the defined 
proportions and finally, the head, with the eyes, will complete the job at each frame. 
For each of these sections, the animators created a multi-dimensional parametric 
blended animations fitting within human constraints to move the proper bones for 
this section all around the required pitch & yaw. 
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The way we built the data for the look at is that we split the body into multiple 
sections and each of these sections are going to be dedicated to a portion of the look 
at. In our case, it is split into 3 parts, the torso, the shoulders/neck and the head. The 
evaluation of the 3D world anchor for the point of interest will be done from the 
bottom to the top. Starting with the torso animation node to compute the 
proportion of its rotation in pitch & yaw that are going to be RTCP internally 
injected. And then the shoulders are going to be computed right after, continuing 
from where the torso stopped and converge a bit more based on the defined 
proportions and finally, the head, with the eyes, will complete the job at each frame. 
For each of these sections, the animators created a multi-dimensional parametric 
blended animations fitting within human constraints to move the proper bones for 
this section all around the required pitch & yaw. 
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Feet Blend Space Computing 

// Get current referential 

CQuaternion CRightFoot ( sourceResult- >m_displacementTransform.GetRotation( ) 

* (modelCommonRaw->GetModelTransfonn( rightFootldx, skeletonRaw, jointTransformArray ) .GetRotation( ) * InvTSRightFoot ) ); 
CQuaternion CLeftFoot ( sourceResult- >m_displacementTransform.GetRctation( ) 

* (modelCommonRaw- >GetModelTransform( leftFootldx, skeletonRaw, jointTransformArray ) .Get Rot at ion ( ) * InvTSLeftFoot ) ); 

CQuaternion feetRot; 

feetRot .QuaternionSlerp( CRightFoot , CLeftFoot, 0.5f ); 
feetRot . RemoveAllExcept Axis ( CVector3( 0.f, 0.f, l.f ) ); 


UpperBack Blend Space Computing 

// Get current referential 

ndQuat upperBack( sourceResult- >m_displacementTransform.GetRotation( ) 

* (modelCommonRaw->Get^odelTransform( upperBackldx, skeletonRaw, jointTransformArray ) .GetRotation( ) * InvTSUpperBack) ); 
m_lookatWorldTransform = ndPosQuatTransform( upperBack, neckPos ); 


We said we built custom spaces for the blending of these stacked sections. In fact of 
all the things we tried, these two are worth to be used : 

- Feet averaged referential for torso and shoulders parts 

- Upper back referential for neck and head parts 

That way it better supports the fact that feet aren't necessary aligned with the 
model space 

To extract them, you simply multiply their transform by the inverse t-stance 
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Data driven look-at layered system 
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VIDEO EXAMPLE 
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Now let's have a look at how it behaves with something moving around the character 

As you can see, with the same data set of parametric animations, it fits properly on 
multiple stances and multiple speeds! You can even polish it more by tweaking the 
weights per situation if you want. 
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Data driven look-at layering for consciousness 


Contextual data scalability 

■ distinction between glance & stare 

■ in car head look-at recycling 

■ anti-look-atfor fire protection 

■ sky is the limit... 

(within animator & memory constraints) 


But this animation node approach can be used to other purposes. We can recycle a 
portion of it and use it in special stances like driving a car. The neck and head data 
will be used but we won't have the full proportion done by the torso since hands are 
mostly used for the steering wheel. We can also use for the opposite of a look at like 
protecting the character from something. 
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Data driven reaction layered system 
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Here is a glimpse of the same animation structure with different data put in it 

protecting the character from some kind of fire 
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Let's follow with the Kinesic track which is supporting the body awareness aspect of 
the game feel. 

Keywords to keep in mind while building these states are connectivity and realism! 
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Contacts with objects for connectivity 
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Enhance the power of touching 
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■ pushing doors 
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■ grabbing grenades 

■ putting on the mask 
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■ getting hands in pockets 
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■ ...or even say hello to strangers! 
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We never have too much contacts with the things on characters while in movement, 
we have to push this forward to boost the immersion factor of the character and its 
body awareness. We mainly did it for simple things like pushing doors, grabbing 
grenades, putting on the mask and putting hands in pockets that enhanced the 
power of touching. This is definitely an area where we could have done better on 
Watch_Dogs though... 
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Multiple recoil layering for realism 



Talking about touching stuff with our hands, on the realism side of things, let's look at 
how firing a weapon was done animation wise, especially on the recoil. 

- First, we exposed some curves in the editor for animators to procedurally animate 
the weapon when it fires 

- Animators had control over moving parts of the weapon and the weapon itself in 
weapon's model space. That way they could synchronize impacts properly and 
because it was procedural, we were able on the code side to accumulate every 
curve playback into an array and stack them for a good feeling of accumulation. 
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Multiple recoil layering for realism 



Then, animators also had curves to drive directly some RTCPs to influence the full 
body parametric animation they created when handling a weapon. That way, we 
could feel the legs counterworking against the power of the weapon. The angle of 
the aiming was purely done by an extra layer of parametric animations integrated 
with the full body pose in a multi-blending node to merge them properly 
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Multiple recoil layering for realism 



And then the camera was also adding noise to the experience to simulate the out of 
control aspect of the weapon firing too much. Designers were able to tweak with the 
same curve tool the noise per weapon how to go vertically and/or horizontally. And 
then the aiming animated pose would adjust based on the camera direction. 
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Multiple recoil layering for realism 


bullets shot 


-t- 1 curves array 


Weapon Skeleton M 


“ hands 
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2-Bone Inverse Kinematic 


Finally, in the post-physics animation update pass, we would complete the job by 
computing the 2-bone IK of both hands to fit the newly placed weapon while taking 
into account the body modifications done by the animations 
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Here is the side view of the character using a rifle 
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This concludes the talk. ..So let's not forget about the player's experience when 
building player mechanics. Always keep in mind usability, grounding, spatial 
awareness and body awareness. I hope you learned interesting tips and I hope you 
will steal them because I stole from a lot of people over the last years... 
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Which leads me to another quote of Frank... 

To thank a lot of people I crossed or worked with to achieve this work. 






For deeper questions, feel free to come and see me tomorrow at the Ubi Lounge! 



